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We are on the precipice of a sea change in 
how research is conducted. Funder mandates and 
changing expectations around _ scientific 
reproducibility have resulted in a new set of 
open science best practices (OSBP) for research. 
These practices include, but are not limited to, 
data and code sharing, —_—shypothesis 
preregistration, registered reports, and open peer 
review. With these changes come challenges and 
potentially unanticipated side-effects. In 
particular, here I will highlight what I see as the 
biggest mismatches between the career 
incentives of the field of neuroscience and 
OSBP, from the perspective of an early career 
researcher. 

Data and code sharing is encouraged or 
required by some journals and funders, but this 
practice has not yet seen widespread adoption - 
“data available upon reasonable request” 
remains the default. For example, a recent 
editorial by an editor who requested data from 
the authors of 41 papers stated that 97% did not 
provide raw data (1). The main limitations to 
data and code sharing are time, training, and 
opportunity cost - which can be offset by 
institutional investment. Keeping code bases up 
to date and data tidy requires training and time 
on the part of students, staff, and researchers. 
The work this takes may limit the rate of 
progress on other activities such as data 
collection or manuscript or grant writing. On the 
other hand, most projects occur in a team 
science environment where good data hygiene 
benefits all. When institutions provide the 
infrastructure to facilitate basic automation 
abilities needed to, for example, process simple 
data without requiring home-brewed code, it 
then becomes easier to share that code in a 
repository. Then, errors can be discovered 
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through reanalysis, other groups can reuse the 
data, the public as well as other scientists trust 
results more when code is available, and data 
faking is disincentivized. 

Preregistration refers to the process of 
outlining hypotheses and analysis plans prior to 
data collection, to prevent bad practices such as 
“hypothesizing after results are known” or 
“p-hacking” (performing repeated tests without 
controlling for false positive rate). Registered 
reports (RRs) are a manuscript format that 
involves peer review of the introduction and 
methods sections of the paper prior to data 
collection. These sections are approved in stage 
1 review, which grants in-principle acceptance 
of the stage 2 manuscript that includes results 
and discussion. The main advantage to 
early-career researchers of RRs is the drastically 
shorter time from review to acceptance - since 
methodological issues are dealt with prior to 
data collection, corrections can be made in 
response to review to an extent that is not 
possible in traditional review. One potentially 
inconvenient reality is that the results of RRs are 
more likely to be negative than positive, and 
without the option to quietly relegate results to 
the file drawer, researchers must accept that 
their favorite theory may not prove correct. 

Open review refers to peer review where 
signed or anonymized reviewer comments are 
shared alongside the published manuscript. The 
benefits of this approach are that the review 
quality and rigor can be _ independently 
evaluated. This may encourage reviewers to be 
more constructive and civil, as adversarial 
reviewers risk their unprofessional behavior 
becoming public knowledge rather than being 
shrouded behind internal journal reviewer 
ratings. Potential downsides include the public 


nature of what is usually a “closed door” critique 
- a bad review going viral could bring unwanted 
attention to the reviewer and authors, and editors 
will need to weigh such outcomes during the 
peer review process. The pressure to write an 
open review may mean it takes longer for 
reviewers to gather their thoughts. For early 
career researchers, the risk of signing a negative 
review from a higher-status researchers’ work 
may be a barrier to participation, and therefore 
ECRs should not be pressured to sign reviews. 
Despite potential downsides, 
reproducibility needs to be a priority for 
neuroscience to maintain integrity of the science 
itself, and to ensure continued buy-in from 
stakeholders such as_ funding agencies, 
politicians, and the lay public whose tax dollars 
fund most research. Recently, multiple scientific 
fields have faced intense backlash and criticism, 
which could likely have been averted if OSBP 
were followed. For example, high-profile social 
psychology findings on “power posing” were 
found to not be replicable (2); the “amyloid 
hypothesis” of Alzheimer’s disease has been 
called into question after decades dominating the 
field (3); and both economics and genetics have 
both suffered pernicious issues with errors 
generated by the use of Excel for data analysis 
(4,5). In the clinical sciences, a starkly low 
compliance rate with the mandate to post 
research study outcomes on clinicaltrials.gov 
highlights the barriers to widespread adoption of 
OSBP (6,7). This is not to mention more 
nefarious issues such as falsifying data, which 
would be automatically disincentivized if data 
and code sharing were universally required. 
Currently, OSBP are not the norm, 
leading the early adopters to sit on what may be 
the initial “bleeding edge” of the coming wave, 
where they assume a higher degree of risk but 
also have the opportunity to be thought leaders 
in their fields. Centers such as the Stanford 
Center for Reproducible Neuroscience, the 
Center for Open Science, and the Open Science 
Foundation are all bellwethers of this change. 
Early career researchers from undergraduates to 
postdocs are in the most ideal position to join the 
second wave “cutting edge” by incorporating 
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OSBP into their work, because their career stage 
involves an expectation of training in new 
techniques. Being on the cutting edge offers 
some of the prestige of the bleeding edge but 
with less risk. Developing a code base can serve 
as part of application portfolios for graduate 
programs or postdocs - computationally savvy 
labs may value coding skills as much as 
“traditional” academic skills like manuscript 
writing or public speaking. Sharing data 
demonstrates commitment to transparency, 
replicability, and robustness of science. 
Preregistration and submission of registered 
reports eliminates the expectation for trainees to 
waste time p-hacking results from data that 
doesn’t support initial hypotheses, and can 
contribute to a record of scientific productivity 
that emphasizes rigor. 

Perhaps the biggest challenge is not the 
implementation of OSBP - the open source tools 
are there, and the training on how to use them is 
largely available online for free. Instead, 
researchers fear that individual career costs 
incurred will outweigh benefits to science. 
Institutions can play an important role in 
offsetting this fear, both by monetarily 
supporting the software, automation processes, 
and other infrastructure necessary for OSBP, but 
also by creating a culture of excellence that 
rewards OSBP (e.g. in tenure review). Public 
response to several recent high-profile 
retractions suggest that attitudes and norms are 
changing in the broader scientific community. 
Nobel prize winner Dr. Frances Arnold retracted 
a paper after data irregularities were found and 
posted a “mea culpa” on Twitter (8). In addition 
to modeling how to graciously admit an error, 
the 5K+ likes on the tweet and 300+ comments 
were largely supportive and even congratulatory, 
suggesting the field finds such transparency 
refreshing. Other researchers have taken a 
similar approach, also with a largely positive 
reception (9), suggesting that slowly but surely 
incentives are changing to better align with 
OSBP. Perhaps when it comes to open science 
best practices, the biggest thing to fear is fear 
itself. 
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